Bayesian Vote Weighting in Crowdsourcing Systems
نویسندگان
چکیده
In social collaborative crowdsourcing platforms, the votes which people give on the content generated by others is a very important component of the system which seeks to find the best content through collaborative action. In a crowdsourced innovation platform, people vote on innovations/ideas generated by others which enables the system to synthesize the view of the crowd about an idea. However, in many such systems gaming or vote spamming as it is commonly known is prevalent. In this paper we present a Bayesian mechanism for weighting the actual vote given by a user to compute an effective vote which incorporates the voters history of voting and also what the crowd is thinking about the value of the innovation. The model results into some interesting insights about social voting systems and new avenues for gamification.
منابع مشابه
Semi-Supervised Consensus Labeling for Crowdsourcing
Because individual crowd workers often exhibit high variance in annotation accuracy, we often ask multiple crowd workers to label each example to infer a single consensus label. While simple majority vote computes consensus by equally weighting each worker’s vote, weighted voting assigns greater weight to more accurate workers, where accuracy is estimated by inner-annotator agreement (unsupervi...
متن کاملWorker Perception of Quality Assurance Mechanisms in Crowdsourcing and Human Computation Markets
Many human computation systems utilize crowdsourcing marketplaces to recruit workers. Because of the open nature of these marketplaces, requesters need to use appropriate quality assurance mechanisms to guarantee high quality results. Previous research has mostly focused on the statistical aspects of quality assurance. Instead, we analyze the worker perception of five quality assurance mechanis...
متن کاملAn Iterative and Re-weighting Framework for Rejection and Uncertainty Resolution in Crowdsourcing
In practical applications of crowdsourcing, labelers may be uncertain or refuse to label a particular instance (or reject) due to the inherent difficulty, and each labeler may be given a different set of instances for big dataset applications. These various issues lead to missing and uncertain labels. Existing crowdsourcing methods have limited capabilities when these two problems exist. In thi...
متن کاملIncreasing the Accuracy of Crowdsourced Information on Land Cover via a Voting Procedure Weighted by Information Inferred from the Contributed Data
Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which they can label cases. Here, the potential to increase the accuracy of crowdsourced data on land cov...
متن کاملPerform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کامل